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Individual Degrees of Freedom for Error 
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This note presents an algorithm based on the Grain-Schmidt orthonormalization pro- 
cedure for producing the coefficients of linear combinations of observations which can be used 
for computing an orthogonal set of individual degrees of freedom for error from a set of ob- 
servations. 

In the analysis of data from designed experiments it is becoming common to compute the 
deviations between the observed and predicted values. Because of the correlation among these 
residuals it is sometimes easier to interpret a set of independent individual degrees of freedom, 
particularly if the residuals are to be used to study the state of statistical control of a measure- 
ment process. This note presents an algorithm for producing such an orthogonal set. 

Let the observations Y u Y 2y . . . Y n have expected values 

E(Y 2 )=l3 Xo2+PiX 12 + . . . +f3 k X k2 

E(Y n )=0 O Xon + PlX ln + . . . +kX kH 
which can be written in matrix notation as 

where y is the vector of observations, X the array of Xt/s and /3 a vector of parameters. 
The deviation, 8 t , between the observed and predicted value is then 

8t=yi — /3 A 0t — ftA H — . . . — PkXa 
or in matrix form 

8=y-X$ 

where p is the vector of estimates of the parameters of the system. Noting that the 0* are 
functions of the observations each 5* can also be written as a function of the observation, say 

8= Ay. 

To say that there are m degrees of freedom for error is to say that .1 has rank m or that the rows 
of A are linear combinations of a set of m orthogonal rows. These m orthogonal rows give 
the coefficients necessary for computing the individual degrees of freedom for error. 

The orthogonalization can be carried out by an application of the Gram-Schmidt process, 1 



i Philip Davis and Philip Rabinowitz. A multiple purpose orthonormalizing code and its uses, Journal of the Association for Comp uting 
Machinery 1, 183-191 (1954). 
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a modification of which follows. The deviations can be written as function of the observations 
as follows: 

8i=a n y 1 +a 12 y2+ . • ■ +a ln y n 
<5 2 =«2i Vi+a 22 y 2 + . • • +a 2n y n 

dn=an l yi + a n2 y 2 J r . . . +a nn y n . 

Consider now the vectors 

ai=(on, a 12 , . . ., a ln ) 

d2 =z \(^2l, &22, • • •, Q>2n) 
Q J n =1 V^nlj Q"n2) • • • > &nn) 

and denote the sums of squares and cross products as follows 

n 
i = l 

Form the vectors 

61= «i 

b 2 = (ai • ai)a 2 — (01 • a 2 )a! 
63= (ai • ai)a 3 — (01 • a 3 )ai 

&„= (^ • ai)a n — (a r a „)ai 

where the vectors b t are the vector sums of the indicated vectors, e.g., the elements of b 2 are 

Oi-ai)a 2 i— (a 1 -a 2 )an, (a 1 -a 1 )a 22 — Oi -a 2 )a 12 , . . ., (ai-aOckn— («i •« 2 )«i«. 
Next form 

c 2 = 62 

c 3 = (62-62)63— (62-63)62 
c 4 =(6 2 -6 2 j64— (6 2 -6 4 )6 2 



Then 



c n = (b r b 2 )b n — (b r b n )b 2 



d 1 = Ci 

d,2= c 2 

d s = c z 

d±=(c r C3)C A — (CyCi)c 3 
d 5 =(c r Ca)c 5 — (c r C 5 )c s 



If in the process one of the vectors becomes all zeros, it should be transferred to the end 
and the vectors renumbered accordingly. 

This process is continued for m steps at which point the last n-m vectors will be zero 
vectors and the first m will be the desired orthogonal set. It is easy to verify that at the Mh 
stage the first (k-1) vectors are mutually orthogonal and orthogonal to all vectors below. 
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To illustrate, consider the design where all differences among four objects are measured 
i.e., 

E(r,)=A-A 
E(rj)=fc-& 

^(F^ft-ft 
#(F s )=ft-ft 
#(F a )=ft-ft, 

The following table shows the estimates of the fts under the assumption that 2/3=0. 

F F 2 F 3 F 4 F 5 F 6 

4ft 1110 

4ft -10 110 

4ft 0-1 0-1 1 

4ft 0-1 0-1-1 



45, = 4F I -4(ft-ft) 

45 2 = 4F 2 -4(ft-ft) 

45 3 = 4F 3 -4(ft-ft) 
45 4 =4F 4 -4(ft-ft) 
45 s = 4F 6 -4(ft-ft) 
45 6 =4F a -4(ft-ft) 

From this table one has, for example: 
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ft=i(F+F 2 +F 3 ) 

5 1 = i(2F 1 -F 2 -F 3 +F 4 +F 5 ) 



The matrix A is then 



<*!=( 2 


-1 


-1 


1 


1 


0) 


a 2 =( — 1 
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— 1 


— 1 





1) 


a 8 =(— 1 


-1 


2 





-1 


-1) 


04= ( 1 


-1 
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-1 


1) 


a 5 = ( 1 





-1 


— 1 


2 


-1) 


a 6 =( 


1 


-1 


1 


-1 


2) 



(aity) = 8 
(aia 2 ) = — 4 
(aia 3 ) = — 4 

(aid 4 ) = 4 
(dia 5 ) = 4 

(aia 6 J = 



The first step gives 



b,= a, -(2 


— 1 


-1 


1 


1 


0) 




6 2 =i(8a 2 +4a 1 ) = (0 


3 


-3 


— 1 


1 


2) 


(6 2 6 2 )= 24 


63=1(8^3+4^) = (0 


-3 


3 


1 


— 1 


-2) 


(6 2 6 3 )=-24 


6 4 =i(8a 4 -4a 1 ) = (0 


— 1 


1 


3 


-3 


2) 


(6 2 6 4 )= - 8 


65=1(805-4^) = (0 


1 


— 1 


-3 


3 


-2) 


(6 2 6 5 ) = 8 


6 6 =i(8a 6 ) = (0 


1 


— 1 


1 


— 1 


2) 


(6 2 6 6 )= 8 



The vectors 6 2 , 6 3 , . . . are multiplied by factors (e.g., 1/4, 1/4, . . .) to keep the numbers 
convenien tly si nail. 
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The second step gives 



The next stage gives 



Cl= 
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246 3 + 246 2 = 


= (0 
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C4 = '5T I 


246 4 + 86 2 ) = 


= (0 
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1) 
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d to the end to give 
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rf 4 =3c 4 +3c 3 = 


= (0 
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d 5 =3c 5 


-3c 3 = 


-(0 














0) 




d&= 


= 


-(o 














0) 





with d 1 , d 2 , d z , being the desired set of orthogonal vectors. 

It is worth noting that one need not start with the deviations, but may begin with any set 
of linear functions having zero expectation and rank equal to the number of degrees of freedom 
for error. For the example the three vectors below could have been used. 



Oi=(l -1 
a 2 =(0 
a 3 =(0 1 -1 



1 0) 
1 -1 1) 
1) 



The steps in the orthogonalization are 



61= 0,!=; 1-1 1 0) 

b 2 =3a 2 -a 1 =(-l 10 2—33) 
6 3 =3a 3 +ai=( 12-3103) 

d= b 1 =( 1 -I 1 0) 

c 2 = 62 =(— 1 1 2 -3 3) 

c 3 =^(24& 3 -126 2 ) = ( 11-2011). 



(Paper 67B1-89) 
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